Unassailable Ideas: How Unwritten Rules and Social Media Shape Discourse in American Higher Education by Ilana Redstone & John Villasenor

Author:Ilana Redstone & John Villasenor [Redstone, Ilana & Villasenor, John]
Language: eng
Format: epub
Tags: social science, sociology, General, education, Higher, Philosophy; Theory & Social Aspects, political science, Comparative Politics, law
ISBN: 9780190078065
Google: PUD8DwAAQBAJ
Publisher: Oxford University Press
Published: 2020-10-15T20:30:54.833673+00:00

8.3 Evaluating Research Quality

The first of the three beliefs is that any action taken to undermine traditional frameworks or power structures is automatically deemed good. One of the many consequences of this belief is that traditional ways of evaluating academic research, and even traditional ways of acquiring and conceptualizing knowledge, can be viewed as suspect. This raises the issue of how academic research is assessed. While emphasis in the popular imagination is often placed on the fact of publishing (as reflected in the “publish or perish” adage), within academia it is just as important to gauge the quality of what is published.

Given the high degree of specialization in most academic fields, it can be hard even for people within the same academic department to read and confidently evaluate the published work of their colleagues. This is a key reason why for academic hiring and for major promotions (such as the granting of tenure), colleges seek letters of reference from outside experts who work in the candidate’s area of research. In addition, the quality of a candidate’s research (for hiring and for major and minor internal promotions) is often assessed using metrics such as the number of publications and through quantitative measures of their impact.

Any attempt to quantify research output is imperfect. Counting the number of papers produced by an academic researcher provides no indication of paper quality. To avoid relying solely on publication count, colleges also try to assess impact. For articles published in academic journals, citation counts (i.e., the number of times the article is cited in future academic publications) are often used.

In addition to attempting to measure the quality of individual papers, academics have also devised metrics aimed at assessing the quality of journals. Journals are often evaluated using their impact factor, which can be defined as “the average number of citations received per article published in that journal during the 2 preceding years”1 (153). All else being equal, a widely cited article appearing in a journal with a high impact factor will be given more weight in hiring and promotion decisions than an article with few citations and/or published in a journal with a low impact factor. For academic books, potential indicators of quality include not only the critical reception the book receives, but also the reputation of the academic press, the number of sales, and (when sufficient time has passed) the number of citations.

All of these metrics are imperfect. Consider a visionary paper that might have its greatest influence (and therefore generate the most citations) multiple years after its original publication. This can mean that many promotion cycles elapse before the value of the paper is recognized. The same phenomenon can limit the utility of metrics for evaluating journal quality. When journal impact factors use a look-back period of only two years, there is no credit given to visionary articles that might have their greatest influence (and therefore generate the most citations) years later.

Despite these flaws, citations and the reputation of the journal or academic presses in which a researcher publishes are widely recognized indicators of the importance of scholarship.

Download

Copyright Disclaimer:
This site does not store any files on its server. We only index and link to content provided by other sites. Please contact the content providers to delete copyright contents if any and email us, we'll remove relevant links or contents immediately.